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H. Gary Cook, Narek Sahakyan, Wisconsin Center for Education Research 
Robert Linquanti, WestEd 


The reauthorization of the Elementary and Secondary Education Act as the Every Student 
Succeeds Act of 2015 defines recently arrived English learners (RA ELs) as EL students who 
have been enrolled in U.S. schools for less than 12 months. For these students, the law permits 
States to select one of two options for including these students in the State’s academic 
achievement accountability determinations. Option 1 excludes RA ELs from taking the required 
reading/language arts (R/LA) assessments in the first year of their enrollment and from any 
accountability determinations based on the R/LA, math, and English language proficiency (ELP) 
assessments; the students’ results on R/LA, math, and ELP assessments, however, are still 
reported. During the second year, these students must be included in the State’s R/LA 
assessment, in R/LA and math achievement indicator calculations, and in progress toward 
achieving ELP indicator calculations. Under option 2, States must assess and report the 
performance of RA ELs on R/LA, math, and ELP assessments in the first year of enrollment. If a 
State chooses this option, it may exclude a RA EL student’s results from the school’s academic 
achievement accountability determination for R/LA and math in the first year of enrollment; for 
a RA EL student’s second year of enrollment, the State must use a measure of RA EL students’ 
academic growth in R/LA and math in accountability determinations; and for the RA EL 
student’s third and succeeding years, the State must include a measure of a RA EL’s proficiency 
in R/LA and math in those determinations. A State could also assign option 1 or 2 (or no option 
at all) based on a RA EL student’s initial English language proficiency level and other possible 
factors, on a statewide basis (termed “option 3” in this paper).' 


The following analyses use a guide published by the U.S. Department of Education on 
RA ELs (Linquanti & Cook, 2017)? to illustrate procedures that can be used to compare and 
contrast school-level overall and EL subgroup accountability determinations for proficiency in 
R/LA under the different options allowed by provisions of the Every Student Succeeds Act. As a 


1 This third option was outlined in final regulations for Elementary and Secondary Education Act of 1965, as 
Amended by the Every Student Succeeds Act — Accountability and State Plans (November 29, 2016) § 
200.16(c)(4). However, those regulations may be overturned by Congress under the Congressional Review Act 
(H.R.J. Res. 57, 2017). The research and analysis in this report were conducted prior to this resolution. Even without 
the Department of Education regulations, however, a State could adopt a statewide procedure to assign option 1 or 2 
to certain categories of RA ELs, with appropriate parameters similar to those outlined in the U.S. Department of 
Education’s regulations. The authors recommend that each State ensure that the student factors it uses are research- 
based, are used statewide for statistical purposes, and do not violate civil rights requirements, which could occur 
with factors such as disability status or nationality. 

* This guide was published prior to passage of the 2017 Congressional Review Act and thus references 
accountability regulations that may be rescinded. However, the guide contains helpful information for States in 
designing their accountability systems with regard to RA ELs, under the relevant statutory provisions. 
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technical reference, the Appendix provides detailed tables and statistical programming code used 
to compute modelled outcomes under the different accountability options. 


These model analyses are provided only to illustrate how a State could undertake them as 
part of its efforts to develop and explore a theory of action for RA EL assessment. The results 
shown here are not in any way intended to recommend or critique any of the options, nor any of 
the calculations used to create them. Several factors limit inferences from these analyses, and no 
generalizations to other states can or should be made. First, these analyses are based on a single 
State’s RA EL population and content and ELP assessment data. States vary greatly in their 
demographic composition of monolingual-English, EL, and RA EL students. Academic content 
assessments and their respective proficiency designations also vary across States, as well as 
across consortia.? Because of this, the RA EL accountability model analyses shown here may 
lead to different outcomes in another State. Second, lack of reliable data on some key variables 
forced us to make several assumptions when determining populations and calculating outcomes 
under different accountability models. States need to pay close attention to the availability and 
uniqueness of their own data elements and systems, and adjust their assumptions accordingly. 
Simply put, results of these analyses will vary—possibly substantially—from State to State. 


In sum, the purpose of these analyses is to provide technical guidance to States on how 
various RA EL accountability options might be enacted. The analyses and results provided below 
are intended to serve as a helpful heuristic—a way of understanding what the analytical task 
entails. Herein, the focus is less on the actual results of these particular analyses and more on the 
analytical methodology used to generate results in light of any particular theory of action that 
states may construct regarding their RA EL accountability model. 


RA EL Accountability Models 


What follows are analytical approaches to three RA EL exception options for R/LA 
assessments and accountability as specified in ESSA’s currently enacted regulations (§ 
200.16(c)(3) & (4)): Option 1 excludes RA ELs from the R/LA test in year 1, administers the 
R/LA test in year 2, and uses results for the R/LA achievement indicator in year 2; option 2 
administers the R/LA test in year 1, administers the R/LA test in year 2, and uses growth for the 
R/LA achievement indicator in year 2; and option 3 assigns RA EL students to option 1 or 2 
based on their initial ELP level. (Note that analyses presented here do not use any other student 
variable other than ELP level to assigned students to option 3. As mentioned earlier, other 
assignment methods are allowed, but they are not applied here.) and possibly other student 
characteristics. All models below represent the calculation of only the second year of R/LA 
performance for RA EL students. 


3 The current analyses are based on test score data from a State that administers the Partnership for Assessment of 
Readiness for College and Careers R/LA and the ACCESS for ELLs 2.0 ELP assessments. A preliminary analysis of 
ACCESS-academic content test relationships provided evidence that ELs’ performance on academic content tests 
varied depending on the administered test (unpublished WIDA board meeting presentation, 2016). 
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Given these three options, six RA EL accountability models are described below. 


1) Option 1 model (option 1): status model. Option 1 provides a baseline or comparative 
option. This option excludes RA ELs from R/LA assessment and accountability in year 1, 
and fully incorporates them in R/LA assessment and achievement calculations in year 2. It is 
simply the number of R/LA proficient students in a group divided by the number of enrolled 
students in that group in a school, including RA ELs. This can be expressed as 


Number of Proficient Students 
Total Number of Students Enrolled 


A potential theory of action for this model is that an unadjusted R/LA percent proficient in 
year 2 (after excluding these ELs from assessment and accountability in year 1) provides a 
more meaningful reflection of RA EL student performance. Calculating without growth or 
any proficient adjustments will motivate schools to support better RA EL outcomes. 


Option 2 models. These three models (specified below) use growth instead of status in 
determining R/LA proficiency for RA ELs. First, a note on interpreting formulae is in order. The 
phrase “total number of students enrolled” refers to the number of enrolled students in specific 
groups (e.g., all students or the EL subgroup) in a school. Option 2 models can be expressed as 


Number of Proficient Students + >} Non Proficient RA ELs making growth 
Total Number of Students Enrolled 


A potential theory of action for these models is that R/LA proficiency per se is not a meaningful 
reflection of RA EL student performance, and that growth in R/LA is a better indicator of RA EL 
performance. Thus, accounting for RA ELs’ growth in R/LA will motivate schools to better 
support RA EL learning. 


With option 2 models, non R/LA proficient RA EL students making acceptable progress (as 
defined by the State) are added to the numerator, and the overall R/LA proficiency results are 
adjusted accordingly. As the formula above indicates, RA ELs making growth targets are 
assigned a “1.” Models 2a—2c below describe different approaches a State could use to determine 
which RA EL students are making acceptable growth, which, in turn, determines which RA EL 
students are credited and included in the numerator in the Option 2 models. Note that the listed 
approaches are by no means exhaustive. 


2) Option 2 — model A (option 2a): value table growth model. This model applies a simple 
value table, defined in Table 1. An English Language Arts (ELA) exam is the R/LA test for 
this particular State. 
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Table 1. Value Table Growth Model 


Year1 ELA Year 2 ELA Proficiency Category 
Proficiency 
Category la Ib lla IIb Illa IIIb >IV 
la 0 1 1 1 1 1 1 
Ib 0 0 1 1 1 1 1 
lla 0 0 0 1 1 1 1 
IIb 0 0 0 0 1 1 1 
Illa 0 0 0 0 0 1 1 
IIIb 0 0 0 0 0 0 1 
>IV 0 0 0 0 0 0 1 


This value table breaks each R/LA proficiency category into two additional categories (e.g., I 
into Ia and Ib). This is done by identifying the median scale score in the range of scale scores 
within a proficient category and labeling scale scores less than the median value as “a” and 
scale scores at or above the median scale score value as “b.” (For more on value tables see 
Hill, 2006). RA ELs who move up one category are assigned a “1.” If a RA EL stays at the 
same category or goes down a category, they are assigned a “0.” RA ELs at performance 
level IV or higher are assigned a “1”; this category in this example represents the State’s 
proficient performance standard in R/LA. 


3) Option 2 — model B (option 2b): percentile growth model. This model applies a percentile 
growth model to determine growth. There are a variety of ways to calculate the percentile 
growth score. One method could be student growth percentiles, as described by Betebenner 
(2011). Given the limitations on both sample size and score histories, a simpler method is 
used here. This simpler model takes the R/LA growth score for all students and ranks them in 
percentiles. RA EL students growing at or above the 40" percentile are considered to make 
the growth target and are assigned a “1.” Otherwise, RA ELs are assigned a “0.” Just to 
reiterate, this model is not student growth percentiles but a simpler variant used for 
illustrative purposes. 


4) Option 2 — model C (option 2c): residual gains growth model. In this model, a post-on-pre 
(R/LA score) regression analysis is conducted for all students. If RA EL students make 
better-than-predicted growth, as contrasted with regression residuals that control for students’ 
grade and ELP level, they are assigned a “1.” Otherwise, RA ELs are assigned a “0.” 


Option 3 models. These models incorporate use of options 1 and 2 based on a RA EL’s 
initial ELP level. A State should to establish a uniform, statewide procedure for determining 
which initial ELP level associates with which option. Accordingly, a State would assign low 
ELP-level RA ELs to one option and high ELP-level RA ELs to the other. The charge for the 
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State then is to decide which ELP level determines the option to which an RA EL is assigned. 
Option 3 models can be expressed as 


Number of Proficient Students + 
Yi non proficient RA ELs making growth (High or Low initial ELP Level) 
Total Number of Students Enrolled 


A potential theory of action for these models is that a RA EL’s initial ELP level affects R/LA 
performance, and effects are different for those at different ELP levels. Accounting for this 
difference better identifies schools that are supporting or not supporting RA ELs, and using these 
models will motivate schools to better support RA EL learning. 


5) Option 3 — model A (option 3a): high-growth/low-status model. This option assigns 
option 1 (baseline) or option 2b based on the student’s initial ELP level. Specifically, RA EL 
students at higher initial ELP levels (> level 4)* receive option 2b (included in assessment in 
year 1 and assigned a “1” if they attained 40th percentile growth or higher in year 2), and 
lower initial ELP-level RA ELs receive option 1 (exempt from R/LA assessment and 
accountability in year 1). 


6) Option 3 — model B (option 3b): low-growth/high-status model. As with option 3a, this 
model uses a combination of options 1 and 2b, but switches the assignment of these options. 
That is, option 3b excludes high-ELP level RA ELs from R/LA assessment and 
accountability in year 1: RA EL students at lower initial ELP levels (< level 4) receive option 
2b, while those at higher initial ELP levels (> level 4) receive option 1. 


Note that option 2b was chosen for option 3 approaches based on ease of programming and 
should not be interpreted to be a better model. Note also that a State could differentiate three 
groups of RA ELs for option 3 models. One group would receive option 1, another option 2, and 
a third neither option. For example, RA ELs at high ELP levels would not receive option 1 or 2. 
They would be fully included in the accountability system. Intermediate ELP level RA ELs 
would receive option 1, and low ELP level RA ELs would receive option 2. 


Ultimately, the objective of these models is to calculate a percentage, which is used to 
support the R/LA academic achievement indicator for ESSA accountability purposes. These 
models can be applied overall at the school-level or for use in school-level EL subgroup 
calculations. Both types of results are calculated and compared in analyses below. Again, note 
that this is neither exhaustive nor necessarily a “best” list of approaches. These examples are 
intended to serve as illustrations of possible analytic approaches and should be considered as a 


4ELP level 4 is used to differentiate model choices for Option 3. This choice is arbitrary but empirically informed. 
The goal is to identify an ELP level sufficiently high (or low) that distinguishes among model choices. For example, 
a State may decide that high-ELP-level RA ELs should be afforded option 1. In determining what constitutes a 
“high” ELP level, States typically employ an amalgam of policy, experience, and empirical information. 
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heuristic. A State’s theory of action should drive the ultimate selection of the RA EL 
accountability model. 


Data 


The following analyses are based on data from a single State that provided longitudinally 
connected, individual-level academic content assessment results for all students, and ELP 
assessment results for all ELs enrolled in State A throughout the academic growth cycles of 
2014-15 and 2015-16. State A’s academic content measure is the Partnership for Assessment of 
Readiness for College and Careers assessment; it also uses the ACCESS for ELLs 2.0 
(ACCESS) as its ELP assessment for ELs. Due to the timing of academic content and ELP 
assessments, and the computational requirements of the different models, only matched score test 
data were used for students enrolled in grades 3-7 in 2015 and in grades 4-8 in 2016.° 
Additionally, given the disproportionately small number of RA EL students in State A (see 
below), as well as for the purpose of economy in this report, school-level accountability models 


were aggregated across grades within schools.® 


The process of identifying the population of RA ELs required utilizing some assumptions. 
Namely, the only way to identify RA ELs in this dataset was to cross-check with the ACCESS 
data available through WIDA’s Data Warehouse, to verify that the first year of test 
administration for these students was 2014-15. Table 2 provides descriptive statistics on 
State A’s English-only, EL, and RA EL students. 


Table 2. English Only, EL and RA EL Students in State A, 2014-15, 2015-16 


2014-15 2015-16 
Mean Mean 
N With ELA ELA* N With ELA ELA 

Student N ELA* Scale | Profi- N ELA Scale Profi- 
Subgroup Total | Score | Score | cient Total Score Score cient 
English Only 286,035 | 285,261 741 40.6% | 291,702 | 291,195 740 40.8% 
English Learner 15,041 | 14,672 701 3.4% 9,374 9,350 699 1.5% 
Recently Arrived 

English Learner 2,709 695 705 8.4% 2,360 2,292 698 5.1% 
(RA EL) 


Table 2 highlights another issue that affects the current analysis. Across the whole State, in 
grades 3-7 (in 2015), the number of RA EL students with valid ELA scores is disproportionately 
smaller than that of English-only students. In 2014-15, there were 15,041 ELs in grades 3-7. Of 
that number 97.5% (14,672) participated in the ELA assessment. Conversely only 25.7% of RA 
ELs (695/2,709) have valid ELA scores, likely due to RA EL exemptions. There is a much larger 
proportion of RA ELs participating in 2015-16 (2,292/2,360 or 97.1%). Accountability models 


° Test scores from the ELA assessment can be reliably compared only for grades 3-8. 
6 Extending the analysis to the grade level is trivial and will not affect the methodology. 
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on RA EL students’ growth in ELA from 2014-15 to 2015-16, such as growth percentiles, value 
tables (based on growth in ELA), and residual gains will, in this example, likely have higher 
error rates. Because the number of RA EL students is so small across the State, one should not 
expect the various school-level accountability models to produce substantially different results in 
this State. Depending upon the concentration of RA ELs in schools, there might not be 
substantially different results across accountability models even for the EL subgroup within 
schools. 


Table 3 displays the distribution of ACCESS ELP levels by EL and RA EL status. Two 
findings are of interest. First, there are more RA ELs at lower ELP levels relative to ELs overall. 
Most RA ELs (56%) are at the lowest three ELP levels, while overall, most ELs (56%) are at the 
highest three ELP levels. Second, RA ELs have greater proportions of students proficient in ELA 
than their non-RA EL peers at virtually every ELP level. At ELP level 5, for example, 3.4% of 
non RA ELs are proficient in ELA. At that same ELP level, 10.1% of RA ELs are proficient. At 
least for State A, it should not be assumed that RA ELs underperform academically compared to 
their EL peers at the same ELP level. Note, however, that the numbers in Table 3 say nothing 
about how long it would take low ELP-level RA EL students to attain higher ELP levels. Do RA 
ELs at ELP level 1 take the same time to attain ELP level 3 as their non RA EL peers? The 
available data cannot address this question. Restated, in this State, RA ELs at particular ELP 
levels perform similarly to their non RA EL peers on this R/LA test; however, it is unknown 
whether these RA ELs’ progress in ELP are at similar rates to their non RA EL peers’ progress. 


Table 3. Number and Percentage ELA and ELP Proficient, by Group in 2015-16 


Non RA English Learners Recently Arrived English Learners 
ACCESS ELP 
ELA At ELP ELA At ELP 
Level N N 

Proficient Level Proficient Level 

1 3 0.0% 1.0% 162 0.0% 7.1% 

2 813 0.2% 8.7% 519 0.0% 22.6% 

3 3,202 0.1% 34.2% 607 0.2% 26.5% 
4 3,531 0.7% 37.8% 504 1.0% 22.0% 
5 1,549 3.4% 16.6% 358 10.1% 15.6% 

6 162 37.7% 1.7% 142 52.8% 6.2% 


Note: ACCESS for ELLs 2.0 specifies six ELP levels: Entering, Emerging, Developing, Expanding, Bridging, and 
Reaching. Additional information about these levels is available at https://www.wida.us/standards/eld.aspx. The 
ELP levels shown are based on the ACCESS for ELLs 2.0 2016 standard setting. 


Tables 4 and 5 show the relationship between ELs and RA ELs, respectively, with and 
without interrupted formal education (SIFE). Table 4 displays this relationship for ELs by ELP 
level. Seventy-nine percent of SIFE students are at the lowest three ELP levels while 46% of 
non-SIFE students are at those same ELP levels. At lower ELP levels, non-SIFE and SIFE 
students do not differentiate by ELA proficiency. SIFE students at ELP level 4 outperform non- 


Recently Arrived English Learners 
SIFE students (non-SIFE = 3.2%, SIFE = 5.8%). However, caution is advised, as there are so 
few SIFE students in State A’s dataset. 


Table 4. Comparison of Non-SIFE and SIFE EL Students’ Proficiency 
in ELA by ELP Level, 2015-16 


Non-SIFE ELs SIFE ELs 

ACCESS Proficient At ELP Proficient At ELP 

ELP Level N ELA Level N ELA Level 
1 385 0.3% 3.6% Td: 0.0% 21.6% 
2 1,188 0.1% 11.1% 104 0.0% 31.7% 
3 3,365 1.2% 31.5% 84 1.2% 25.6% 
4 5,684 3.2% 53.2% 69 5.8% 21.0% 
5 69 2.9% 0.6% 0 -- 0% 
6 0 -- 0% 0 id 0% 


Note: ACCESS for ELLs 2.0 specifies six ELP levels: Entering, Emerging, Developing, Expanding, Bridging, and 
Reaching. Additional information about these levels is available at https://www.wida.us/standards/eld.aspx. The 
ELP levels shown are based on the ACCESS for ELLs 2.0 2016 standard setting. 


Table 5 displays, in matrix form, differences in ELA proficiency among SIFE and non-SIFE, 
and RA EL and non-RA EL students. Comparing RA EL groups, non-SIFE RA ELs demonstrate 
a higher rate of ELA proficiency than their SIFE RA EL counterparts (5.3% versus 0.7%, 
respectively). However, as seen in Table 4, SIFE students tend to cluster at lower ELP levels, 
which systematically relates to EL students’ ELA proficiency. That said, in comparing non-RA 
EL groups, SIFE non-RA ELs demonstrate a slightly higher rate of ELA proficiency (2.2%) than 
their non-SIFE non-RA EL counterparts (1.5%), the latter also being by far the most common 
permutation.’ 


Table 5. EL Students’ Proficiency in ELA by SIFE and RA EL Status 


SIFE Non-SIFE 

0.7% 5.3% 
RA EL (N=147) (N=1,730) 

2.2% 1.5% 
Non-RA EL (N=181) (N=8,967) 


In sum, in State A, SIFE status appears to differentiate performance among RA EL students, 
but less so among non-RA EL students. As might be expected, SIFE students also tend to cluster 
at lower ELP levels, which may affect SIFE RA ELs in particular. However, due to the small 


7 Non-RA EL students likely include many long-term ELs, a status that often predicts academic underperformance 
and might override the effects of non-SIFE status on non-RA ELs’ ELA performance. 
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number of SIFE students in this sample, SIFE and non SIFE students will not be differentiated in 
the following RA EL models analyses. 


Results of RA EL Accountability Model Analyses 


Tables 6 and 7 summarize the results of the six models described above, first for school 
outcomes based on all students, then for school outcomes for the EL subgroup, each 
disaggregated by school concentration of RA ELs. Disaggregating by RA EL concentration is 
important because RA ELs tend to be at lower ELP levels, as seen in Table 3. Given that ELs’ 
ELP level is related to R/LA performance, there may be distinctions between accountability 
models based on RA EL concentrations. The Appendix contains detailed descriptive statistics 
and inter-model correlations for the EL-subgroup results by each level of school concentration of 
RA ELs. 


Table 6. Number and Percentage of All Students ELA Proficient, Based on RA EL Model 
Option, by School Concentration of RA ELs 


Number and Percentage of ALL Students ELA proficient 

RA EL : : : : 
Model Schools With Any Schools With Schools With Schools With 
Option RA EL student 1 to 9 RA ELs 10 to 49 RA ELs 50 or more RA ELs 

N % N % N % N % 
Option 1 452 36.63% 375 37.40% 74 32.63% 3 39.71% 
Option 2a 452 36.73% 375 37.48% 74 32.82% 3 39.69% 
Option 2b 452 36.81% 375 37.54% 74 33.00% 3 39.79% 
Option 2c 452 36.76% 375 37.50% 74 32.88% 3 39.68% 
Option 3a 452 36.78% 375 37.51% 74 32.90% 3 39.78% 
Option 3b 452 36.75% 375 37.50% 74 32.85 % 3 39.92% 


Options: 1 = status baseline model, 2a = value table growth model, 2b = percentile growth model, 2c = residual 
gains growth model, 3a = high-growth/low-status model, 3b = low-growth/high-status model 


Table 6 displays the number and percentage of all students (EL and non-EL) within schools 
scoring proficient in ELA under each of the RA EL model options, differentiated by schools with 
any RA EL students, as well as broken out by schools with different concentrations of RA ELs. 
As can be seen, there is little difference among the percentages of students identified as ELA 
proficient among RA EL model options. 


All correlations among RA EL accountability model options when all students are in the 
accountability model, including RA ELs, are greater than 0.999 (i.e., r > 0.999; see Appendix, 
Table A.2). Here correlations are among the percentages of students deemed ELA proficient, 
including RA ELs, by each model. For State A, then, at the overall school level, there is no 
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appreciable difference among different RA EL model options regarding the percentage of 
students identified as ELA proficient. 


Table 7 displays the number and percentage of the EL student subgroup (RA EL and non- 
RA EL) scoring proficient in ELA under each of the RA EL model options, differentiated by 
schools with any RA EL students as well as broken out by schools with different concentrations 
of RA ELs. A subgroup minimum N size of 30 was used in determining which schools were 
included in the analysis. This N size reflects the State’s N size rule and reduced the number of 
schools included in this analysis.® 


Table 7. Number and Percentage of EL Student Subgroup ELA Proficient based on RA EL Model 
Option by School Concentration of RA ELs 


Number and Percentage of EL Students ELA proficient 

hee Schools With Any Schools With Schools With Schools With 
Option RA EL student 1 to 9 RA ELs 10 to 49 RA ELs 50 or more RA ELs 

N % N % N % N % 
Option 1 411 3.6% 345 3.8% 63 2.3% 3 1.3% 
Option 2a 411 5.0% 345 5.3% 63 3.4% 3 1.2% 
Option 2b 411 6.8% 345 7.2% 63 4.7% 3 2.2% 
Option 2c 411 5.7% 345 6.1% 63 4.0% 3 1.5% 
Option 3a 411 5.3% 345 5.7% 63 3.8% 3 1.8% 
Option 3b 411 6.4% 345 6.8% 63 4.3% 3 2.6% 


Options read: 1= status baseline model, 2a = value table growth model, 2b = percentile growth model, 
2c = residual gains growth model, 3a = high-growth/low-status model, 3b = low-growth/high-status model 


In looking at percent ELA proficient for the EL subgroup, it is apparent that a much lower 
percentage of students meets the ELA performance standard compared to the all-student results 
shown in Table 6. There are greater differences among RA EL model options. However, the 
maximum difference between baseline option 1 and other options is 3.4 percentage points (the 
difference between option 1 and 2b for schools with one to nine RA ELs). The option 2b model 
has slightly greater percentages of EL students meeting the standard for ELA proficient in all 
RA EL school concentrations, except schools with 50 or more RA ELs. Schools with larger 
numbers of RA ELs do not perform as well; however, this finding can largely be attributed to the 
higher proportion of low-ELP-level EL students among RA ELs. 


The correlations among the models’ percentages of students deemed proficient are also 
slightly lower (see the Appendix, Table A.10). Option 2 models correlate highly with one 
another (between 0.80 and 0.96). The option 1 model correlates with option 2 models at much 


8 This EL student subgroup analysis does not include any former ELs, even though provisions of the Every Student 
Succeeds Act permit states to include the performance of former ELs on statewide ELA and math assessments in the 
EL subgroup (for up to 4 years after the students exit EL status) for Title I accountability. 
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lower levels, ranging from around 0.3 to 0.6. Option 3 models correlate more with option 1 than 
with option 2 models. 


While percentages of students identified as ELA proficient differ among RA EL models in 
State A, those differences are small. This finding drives home the need for this State to clearly 
articulate its theory of action regarding RA EL inclusion in its accountability system. 


Summary 


The analyses presented in this paper are intended to serve two purposes. The first is to 
provide examples of RA EL accountability models that are consistent with federal law and 
operating guidance as of March 2017. With three allowable options, six models are described 
and summarized in Table 8. It is important to note again that these six models are in no way 
exhaustive. Rather, they are meant to serve as a heuristic illustrating how States may use them in 
their theory building for RA EL accountability models. 


Table 8. Described RA EL Accountability Options and Models 


Option Description Federal Reference 

Option 1 Status baseline 

Option 2a Value table growth Every Student Succeeds Act 

Option 2b Percentile growth §1111(b)(3)(A)(i) 

Option 2c Residual gains growth 

Option 3a Combined: options 2b and 1 — high- 
growth/low-status 

Option 3b Combined: option 1 and 2b — low- Sa CER 200: TIENT: 
growth/high-status 


The second purpose is to apply these six models to a State’s actual dataset and explore the 
outcomes. As stated earlier, the application of these models is intended to test the viability of 
ideas and/or potential theories of action that support RA EL accountability models. States should 
not assume that outcomes described here would be similar to those found with RA ELs in their 
schools. For this reason, statistical analysis code is provided in the Appendix to support 
individual State exploration. Ultimately, our purpose has been to spur States to discuss how best 
to serve RA ELs in state accountability systems and to provide some ideas, resources, and tools 
to accomplish this. 


° Based upon H.R.J. Res. 57, 2017, this regulation may no longer be in force sometime after February 2017. 
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Appendix. Detailed Analysis Tables (and Programming Code for Analyses) 


Option 1: status baseline model 
Percentage proficient in ELA with RA ELs included — baseline for comparisons 


Option 2a: value table growth model 
Percentage proficient in ELA for students in subgroup and a value table for RA EL students 


Option 2b: percentile growth model 
Percentage proficient in ELA for students in subgroup and percentile growth for RA EL students 


Option 2c: residual gains growth model 
Percentage proficient in ELA for students in subgroup residual gains for RA EL students 


Option 3a: high-growth/low-status model 
Percentage proficient in ELA for students in subgroup and high-ELP-level RA ELs (overall ELP 


> 4.0) have option 2b and low-ELP-level RA ELs have option 1 


Option 3b: low-growth/high-status model 
Percentage proficient in ELA for students in subgroup and high-ELP-level RA ELs (overall ELP 


> 4.0) have option 2b and low-ELP-level RA ELs have option 1 
A. Detailed Analysis Tables 


All Students in All Schools with RA ELs: 


Table A.1. Descriptive Statistics for All Students ELA Proficient, in All Schools with RA ELs, by 
RA EL Accountability Model Option 


Average SD Percent 
N of Schools Proficient Proficient Min Max 
Option 1 452 36.6% 0.16226 0% 87.8% 
Option 2a 452 36.7% 0.1612 0% 87.8% 
Option 2b 452 36.8% 0.16069 0% 87.8% 
Option 2c 452 36.8% 0.16096 0% 87.8% 
Option 3a 452 37.0% 0.16199 0% 87.8% 
Option 3b 452 36.8% 0.16195 0% 88.2% 


The table reads: Descriptive statistics for the percent of all students in schools with RA ELs who 
attain proficiency in ELA, based on different RA EL accountability models. 
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Table A.2. Correlations of ELA Proficiency Determinations for All Students ELA Proficient, 
in All Schools with RA ELs, by RA EL Accountability Model Option 


Option 1 Option 2a | Option2b | Option 2c | Option 3a | Option 3b 


Option 1 1.00000 -- -- -- << 2s 


Option 2a 0.99969 1.00000 -- -- -- -- 
Option 2b 0.99947 0.99984 1.00000 -- -- -- 


Option 2c 0.9996 0.99991 0.99994 1.00000 -- -- 


Option 3a 0.9978 0.99974 0.99974 0.99975 1.00000 -- 


Option 3b 0.9987 0.99973 0.99965 0.99972 0.99971 1.00000 


The table reads: The Pearson correlations represent the percentage of all students in schools 
with RA ELs who attain proficiency in ELA based on different RA EL accountability models. 


B. All Students in Schools with 1 to 9 RA ELs: 


Table A.3. Descriptive Statistics for All Students ELA Proficient, in Schools with 1 to 9 RA ELs, 
by RA EL Accountability Model Option 


Average SD Percent 
N of Schools Proficient Proficient Min Max 
Option 1 375 37.4% 0.16454 0% 87.8% 
Option 2a 375 37.5% 0.16367 0% 87.8% 
Option 2b 375 37.5% 0.16330 0% 87.8% 
Option 2c 375 37.5% 0.16346 0% 87.8% 
Option 3a 375 37.5% 0.16435 0% 87.8% 
Option 3b 3/5 37.49% 0.16409 0% 87.77% 


Table A.4. Correlations of ELA Proficiency Determinations for All Students ELA Proficient, 
in Schools with 1 to 9 RA ELs, by RA EL Accountability Model Option 


Option1 | Option2a | Option2b | Option 2c | Option 3a | Option 3b 
Option 1 1.00000 -- = - ae ed 
Option 2a 0.99973 1.00000 -- = ae _- 


Option 2b 0.99961 0.99989 1.00000 -- -- -- 


Option 2c 0.99966 0.99992 0.99996 1.00000 -- -- 


Option 3a 0.99985 0.99974 0.99976 0.99974 1.00000 -- 


Option 3b 0.99989 0.99974 0.99971 0.99973 0.99973 1.00000 
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All Students in Schools with 10 to 49 RA ELs: 


Table A.5. Descriptive Statistics for Proficiency Determinations for All Students ELA Proficient, 


in Schools with 10 to 49 RA ELs, by RA EL Accountability Model Option 


Average SD Percent 
N of Schools Proficient Proficient Min Max 
Option 1 74 32.6% 0.14647 9.5% 70.4% 
Option 2a 74 32.8% 0.14459 9.5% 70.0% 
Option 2b 74 33.0% 0.14359 9.7% 69.9% 
Option 2c 74 32.9% 0.1443 9.5% 69.9% 
Option 3a 74 32.9% 0.14517 9.5% 70.4% 
Option 3b 74 32.8% 0.14518 9.6% 70.4% 


Table A.6. Correlations of ELA Proficiency Determinations for All Students ELA Proficient, in 
Schools with 10 to 49 RA ELs, by RA EL Accountability Model Option 


Option1 | Option2a | Option2b | Option2c | Option 3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a 0.99948 1.00000 -- -- -- -- 
Option 2b 0.99860 0.99948 1.00000 -- -- -- 
Option 2c 0.99923 0.99986 0.99984 1.00000 -- -- 
Option 3a 0.99836 0.99967 0.99963 0.99977 1.00000 -- 
Option 3b 0.99977 0.99964 0.99929 0.99964 0.99956 1.00000 


All Students in Schools with 50 or More RA ELs: 


Table A.7. Descriptive Statistics for Proficiency Determinations for All Students ELA Proficient, 


in Schools with 50 or more RA ELs, by RA EL Accountability Model Option 
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Average SD Percent 
N of Schools Proficient Proficient Min Max 
Option 1 3 39.7% 0.13898 24.2% 51.1% 
Option 2a 3 39.7% 0.13728 24.4% 51.0% 
Option 2b 3 39.8% 0.13573 24.7% 51.% 
Option 2c 3 39.7% 0.13542 24.6% 50.8% 
Option 3a 3 39.8% 0.13887 24.3% 51.2% 
Option 3b 3 39.92% 0.13775 24.6% 51.3% 
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Table A.8. Correlations of ELA Proficiency Determinations for All Students ELA Proficient, in 
Schools with 50 or more RA ELs, by RA EL Accountability Model Option 


Option1 | Option2a | Option2b | Option2c | Option 3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a 0.99998 1.00000 -- -- -- -- 
Option 2b 0.99997 1.00000 1.00000 -- -- -- 
Option 2c 1.00000 0.99999 0.99998 1.00000 -- -- 
Option 3a 1.00000 1.00000 0.99847 0.99999 1.00000 -- 
Option 3b 1.00000 1.00000 1.00000 0.99999 0.99999 1.00000 


EL Subgroup Students in All Schools with RA ELs: 


Table A.9. Descriptive Statistics for EL Subgroup Students ELA Proficient, in All Schools with 
RA ELs, by RA EL Accountability Model Option 


Average SD Percent 

N of Schools Proficient Proficient Min Max 
Option 1 411 3.6% 0.09632 0% 100% 
Option 2a 411 5.0% 0.10029 0% 100% 
Option 2b 411 6.8% 0.12208 0% 100% 
Option 2c 411 5.7% 0.10754 0% 100% 
Option 3a 411 5.4% 0.11423 0% 100% 
Option 3b 411 6.4% 0.11855 0% 100% 


Table A.10. Correlations of ELA Proficiency Determinations for EL Subgroup Students ELA 
Proficient, in All Schools with RA ELs, by RA EL Accountability Model Option 


Option 1 Option 2a | Option2b | Option2c | Option3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a 0.56965 1.00000 -- -- -- -- 
Option 2b 0.42881 0.80835 1.00000 -- -- -- 
Option 2c 0.51911 0.93633 0.85632 1.00000 -- -- 
Option 3a 0.80560 0.53654 0.67099 0.51553 1.00000 -- 
Option 3b 0.81066 0.75547 0.69849 0.78042 0.65410 1.00000 
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EL Subgroup Students in Schools with 1 to 9 RA ELs: 


Table A.11. Descriptive Statistics for EL Subgroup Students ELA Proficient, in Schools with 1 to 
9 RA ELs, by RA EL Accountability Model Option 


Average SD Percent 

N of Schools Proficient Proficient Min Max 
Option 1 345 3.8% 0.10371 0% 100% 
Option 2a 345 5.3% 0.10827 0% 100% 
Option 2b 345 7.2% 0.13175 0% 100% 
Option 2c 345 6.1% 0.11602 0% 100% 
Option 3a 345 5.7% 0.12295 0% 100% 
Option 3b 345 6.8% 0.12790 0% 100% 


Table A.12. Correlations of ELA Proficiency Determinations for EL Subgroup Students ELA 


Proficient, in Schools with 1 to 9 RA ELs, by RA EL Accountability Model Option 


Option1 | Option 2a | Option2b |) Option2c | Option3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a 0.56827 1.00000 -- -- -- -- 
Option 2b 0.42708 0.80632 1.00000 -- -- -- 
Option 2c 0.51795 0.93614 0.85356 1.00000 -- -- 
Option 3a 0.80287 0.53093 0.57607 0.50909 1.00000 -- 
Option 3b 0.80941 0.75448 0.69601 0.77959 0.64827 1.0000 


EL Subgroup Students in Schools With 10 to 49 RA ELs: 


Table A.13. Descriptive statistics for proficiency determinations for EL Subgroup Students ELA 
Proficient, in Schools with 10 to 49 RA ELs, by RA EL Accountability Model Option 


Average SD Percent 
N of Schools Proficient Proficient Min Max 
Option 1 63 2.3% 0.038 0% 24.0% 
Option 2a 63 3.4% 0.03299 0% 16.0% 
Option 2b 63 4.7% 0.03983 0% 16.7% 
Option 2c 63 4.0% 0.03646 0% 16.0% 
Option 3a 63 3.8% 0.04503 0% 24.0% 
Option 3b 63 4.3% 0.04002 0% 24.0% 
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Table A.14. Correlations of ELA Proficiency Determinations for EL Subgroup Students 
ELA Proficient, in Schools with 10 to 49 RA ELs, by RA EL Accountability Model Option 


Option 1a | Option 2a | Option2b | Option2c | Option3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a 0.55281 1.00000 -- -- -- -- 
Option 2b 0.38874 0.85929 1.00000 -- -- -- 
Option 2c 0.47714 0.92556 0.96607 1.00000 -- -- 
Option 3a 0.88549 0.70731 0.65887 0.71244 1.00000 -- 
Option 3b 0.84305 0.72835 0.73155 0.75347 0.85673 1.00000 


EL Subgroup Students in Schools with 50 or more RA ELs: 


Table A.15. Descriptive Statistics for Proficiency Determinations for EL Subgroup Students ELA 
Proficient, in Schools with 50 or more RA ELs, by RA EL Accountability Model Option 


Average SD Percent 

N of Schools Proficient Proficient Min Max 
Option 1 3 1.3% 0.01501 0.00% 3.0% 
Option 2a 3 1.2% 0.00412 0.99% 1.7% 
Option 2b 3 2.2% 0.01094 0.99% 3.0% 
Option 2c 3 1.5% 0.00806 0.99% 2.4% 
Option 3a 3 1.2% 0.01040 1.00% 2.0% 
Option 3b 3 2.6% 0.00750 2.0% 3.5% 


Table A.16. Correlations* of ELA Proficiency Determinations for EL Subgroup Students 
ELA Proficient, in Schools with 50 or more RA ELs, by RA EL Accountability Model Option 
(*correlations are questionable, given small sample size) 


Option 1 Option 2a | Option2b | Option2c | Option3a | Option 3b 
Option 1 1.00000 -- -- -- -- -- 
Option 2a | -0.19224 1.00000 -- -- -- -- 
Option 2b -0.9749 0.40589 1.00000 -- -- -- 
Option 2c | -0.18363 0.99996 0.39787 1.00000 -- -- 
Option 3a 0.98532 -0.35696 -0.99860 -0.34876 1.00000 -- 
Option 3b 0.99634 -0.27537 -0.99036 -0.26693 0.99630 1.00000 
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C. Programming Code for Analyses 
*THE FOLLOWING CODE COMPARES SCHOOL-LEVEL ACCOUNTABILITY MODELS IN THE CONTEXT OF RA ELs; 


* You must first run the programming call to acquire the State’s dataset. The current dataset looks at growth 
across the 2014-15 and 2015-16 school years. The 2014-15 school year is designated as 2015, and the 2015- 
16 school year is designated 2016; 


RRRAKKAK KEKE KE Dron unused variables and rename columns for easier reference*********; 
data A; set A; /* A=dataset label for retrieving data for State X */ 
keep student_id 
school_number_2015 
school_name_2015 
school_number_2016 
school_name_2016 
sife_status_2015 
sife_status_2016 
ccr_grade_2015 
ccr_grade_2016 
first_access_year /* designation for RA EL */ 
access composite _scale_score_201 /* ELPA overall scale score */ 
access _composite_pl_2015 /* ELPA overall proficiency level */ 
access_composite_scale_score_200 
access_composite_pl_2016 
ccr_ela_scale_score_2015 /* Career and College ready ELA scale score */ 
ccr_ela_proficiency_level_2015 /* Career and College ready ELA proficiency level */ 
ccr_ela_proficient_2015 /* Proficiency dichotomous variable */ 
ccr_ela_scale_score_2016 
ccr_ela_proficiency_level_2016 
ccr_ela_proficient_2016; 
* these are the only variables that will be used for the analysis; 


run; 
data A; set A; * Renaming variables in dataset; 

rename ccr_grade_2015 = grade15; * Student's grade in 2014-15; 
rename ccr_grade_2016 = grade16; * Student's grade in 2015-16; 


rename access _composite_scale_score_201 = CSS15; * Student's ACCESS Composite Scale Score in 2014-15; 
rename access _composite_scale_score_200 = CSS16; * Student's ACCESS Composite Scale Score in 2015-16; 
rename access _composite_pl_2015 = CPL15; * Student's ACCESS Composite Proficiency in 2014-15; 
rename access _composite_pl_2016 = CPL16; * Student's ACCESS Composite Proficiency in 2015-16; 
rename ccr_ela_scale_score_2015=ELA_SS 15; * Student's ELA Scale Score in 2014-15; 

rename ccr_ela_scale_score_2016=ELA_SS 16; * Student's ELA Scale Score in 2015-16; 

rename ccr_ela_proficient_2015 = ELA_Prof_15; * Student's ELA Proficiency status in 2014-15; 

rename ccr_ela_proficient_2016 = ELA_Prof_16; * Student's ELA Proficiency status in 2015-16; 

run; 


*Reformat ELA Scale Score Variables as numeric; 

data A; set A; 

ELA _SS_ 15 n=input(ELA_SS 15, 8.); ELA SS 16 n=input(ELA_SS_ 16, 8.); 
drop ELA_SS 15 ELA SS 16; 
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rename ELA_SS_ 15 n=ELA_SS_ 15; rename ELA_SS_16 n=ELA_SS 16; 
run; 


* This analysis only uses grades 3-7 in 2014-15 and 4-8 in 2015-16; 
* Identify RAELs (First Access in 2014-15), and drop 2015-16 RAELs; 


data State_X; set A; 

if grade15 LT 3 OR grade15 GT 7 then delete; if grade16 LT 4 OR grade16 GT 8 then delete; 

if first_access_year = '2015' AND CSS15 NE. then RAEL = 1; else RAEL = 0; 

if first_access_year = '2016' AND CSS16 NE. then delete; * ELs that just started in 16 don't count; 
if CPL15 =. then ELL15 = 0; else ELL15 = 1; if CPL16 = . then ELL16 = 0; else ELL16 = 1; 

if CPL16 =. and ELA SS _16 NE. then E016 = 1; else E016 = 0; /* Here EO is former ELs and EOs */ 
if RAEL = 1 then ELL16 = 0; * separate ELs and RAELs into non-overlapping groups; 

run; 


2 ee 2 2 2 ee ee ee ee RK EE EEK 5 rig ble Definitions* * * ** * * ¥ * * ¥ # # FF FF HH KKK KEK K KK. 
, 


Data State_X; set State_X; 

Ivl_change = 0; * RAEL weights based on ELA Growth (Value tables); 
RAEL_pctl_change=0; * RAEL weights based on ELA Growth (40th Percentile); 
RAEL_perf_resid=0; * RAEL weights based on ELA Growth (Residual Gains Model); 


RAEL_prof = 0; * RAELs that are ELA-proficient in 2015-16; 
RAEL_ge4 =0; * RAELs that are CPL 4.0 and above in 2014-15; 
RAEL_It4 = 0; * RAELs that are below CPL 4.0 in 2014-15; 


RAEL_Change_NPr_GE4 =0; * RAELs that changed levels, are above CPL 4.0 in 2014-15 & not ELA proficient 
in 2015-16; 

RAEL_Change_NPr_LT4=0; * RAELs that changed levels, are below CPL 4.0 in 2014-15 & not ELA proficient in 
2015-16; 

RAEL_Pr_GE4 = 0; * RAELs that are above 4.0 CPL & ELA proficient in 2015-16; 

RAEL_Pr_LT4 =0; * RAELs that are below 4.0 CPL & ELA proficient in 2015-16; 

Run; 


[EREPEELES ASLAN ELE LERES ELE EERE EE EEAL EER LER EAE E SELES L EERE ERS SELLERS EERE ER ESE Sf: 


/* OPTION 2a: Level Change in ELA (Value Tables/Linear Growth) */ 
if */ 


[ERESEAAES EELASELEL LER ELE FESS EL LEEEL EES SEARLE EA ELSES LEER ES ELLE ESLER SERA EES ES |. 


*Define Level Change conditional on growth in ELA (PARCC assessment); 

data State_X; set State_X; 

*Start at 1a (650-674.5), move up; 

if ELA_SS_15GE650 ANDELA_SS_15 LE 674.5 AND ELA_SS_16 GT 674.5 then lvl_change = 1; 
*Start at 1b (674.5-699), move up; 

If ELA_SS_15 GT 674.5 AND ELA_SS_15 LE 700 AND ELA_SS_16GT 700 then lIvI_change = 1; 
*Start at 2a (699 - 712), move up; 

If ELA_SS_15GT 700 ANDELA SS _15LE712 AND ELA_SS_16GT712 then Ivl_change = 1; 
*Start at 2a (712 - 724), move up; 

If ELA_SS_15GT712 ANDELA SS 15LE724 AND ELA_SS_16GT724 then Ivi_change = 1; 
*Start at 3a (724 - 737), move up; 

If ELA_SS_15GT724 AND ELA_SS_15LE737 AND ELA_SS_16 GT 737 then lIvl_change = 1; 
*Start at 3b (737 - 749), move up; 

If ELA_SS_15 GT 737 ANDELA_SS_15LE749 AND ELA_SS_16GT 749 then Ivl_change = 1; 
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if ELA_SS_15 GE 750 then Ivil_change = 1; 
run; 


*Apply the (half) Level Changes to RAELs; 

data State_X; set State_X; 

*RAEL_Ivl_change = 1 for RAEL students that have moved up a (half) level; 
RAEL_Ivl_change = RAEL*lvl_change; 

run; 


[EREAARACESER LASER AE EASES REEL SELL LE ERLE SAL EES LEEE ELAR ELE ER EL SEALE EA AL ELE EEE A EEE 7. 


/* OPTION 2b: Percentile Growth in ELA */ 
1a */ 


[EREEAPASESEA SEER ERS SARE SRLS ERASE SS EERE LS LEE RE ELLA REA EELS ASL EEEE EEA SL ESALLE ELE | 


data State_X; set State_X; 

*Compute ELA scale score difference between 2014-15 and 2015-16 ELA; 
ELA_SS_diff = ELA_SS_16-ELA_SS 15; 

run; 

* Preparing for ranking procedure and ranking. 

proc sort data = State_X; by grade16; run; 


proc rank data = State_X groups=100 out = State_xX; 
*Rank the ELA scale score differences; 

var ELA_SS_diff; by grade16; ranks rank_ELA_growth; 
run; 


data State_X; set State_X; 

*Criteria is set at the 40th percentile; 

if RAEL = 1 and rank_ELA_growth GT 40 then RAEL_pctl_change = 1; 
else RAEL_pctl_change = 0; 

*define RAELs that were proficienct in 2016; 

if RAEL = 1 and ELA_Prof_16 = 1 then RAEL_Prof = 1; else RAEL_prof = 0; 

*define RAELs below and above 4.0 CPL; 

if RAEL = 1 AND CPL15 GE 4 then RAEL_ge4 = 1; else RAEL_ge4 = 0; 

*define RAELs below and above 4.0 CPL; 

if RAEL = 1 AND CPL15 LT 4 then RAEL_It4 = 1; else RAEL_It4 = 0; 

run; 


[ER REEFERER EEE EELESE SEER EL ERESALE SELES ELE SE ELER EES EL ERR ELL L EEE EEE A ELE EEE REL ERS EEE | 


1 hes OPTION 2c: Growth in ELA (Residual Gains Model) */ 
‘hd si 


[EREEFERER ERLE APL ES EEE TELLER ESA ES ER ERS ESE SE EL ERE ER ES ERERELES EER ELE RENE ELE REL EES ESE | 


data State_X_reg; Set State_X; 

CPL15 = int(CPL15); 

*English Only and Former EL students receive a CPL of 5.0 a presumed proficient score; 
if CPL15 =. and ELA_SS_15 NE. then CPL15 = 5.0; 

run; 


*Residual Gains Model; 
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proc reg data = State_X_reg; 

*Estimate Pre on Post, controlling for Grade and CPL; 

model ELA_SS_16=ELA_SS 15 Grade15 CPL15; *controlling for starting grade and ELP level; 
output out = State_X_reg_outr =yresid p = predict; 

run; quit; 


data State_X_reg out; set State_X_reg_ out; 

*if performed higher than model average (=0), then 1, else 0; 
if yresid GE O then perform = 1; 

else if yresid LT 0 then perform = 0; else perform = .; 

run; 


data State_X_reg out; set State_X_reg_ out; 
RAEL_perf_resid = RAEL * perform; 
run; 


*Join the datatsets by student ID; 

proc sort data = State_X; by student_id; run; 

proc sort data = State_X_reg_ out; by student_id; run; 

data state_X_join; merge state_X state_X_reg_ out; by student_id; run; 
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[EPEEAAAELL ER EE PEAR ELE EASELS EEL ERS LELEE LEH ELLE REL RA LES AR LEE EL ALES ELSE ERA EEE | 


ia OPTION 3a & 3b: Hybrid.*/ 
i oe 


[EPEEAAAELS ERASE EAR LES EE RERERE EEL ER SL ELLE LEH EER A EEL ERA EES ARLE EES EL EL EL ELSE ERA EEE 


data State_X_join; set State_X_join; 


*Option3a; 

*High ELP->2b, Low ELP -> Baseline; 

If RAEL = 1 AND CPL15 GE 4 AND RAEL_Prof NE 1 then RAEL_Gr_NPr_GE4 = RAEL_pctl_change; 

else RAEL_Gr_NPr_GE4 = 0; *Count High-level non-proficient RAELs that made growth (1 if >40th%); 
IF RAEL = 1 AND CPL15 LT 4 AND RAEL_Prof EQ 1 then RAEL_Pr_LT4 = 1; 

else RAEL_Pr_LT4 = 0; *Low-level proficienct RAELs; 

IF RAEL = 1 AND RAEL_Prof NE 1 AND CPL15 LT 4 then RAEL_NPr_LT4 = 1; 

else RAEL_NPr_LT4 = 0; *Low-level non-proficient RAELs; 


*Option3b; 

*High ELP level -> Baseline, Low ELP Level -> 2b; 

If RAEL = 1 AND CPL15 LT 4AND RAEL Prof NE 1 then RAEL_Gr_NPr_LT4 = RAEL_pctl_change; 

else RAEL_Gr_NPr_LT4 = 0; *Count Low-level non-proficient RAELs that made growth (1 if >40th%);; 
IF RAEL = 1 AND CPL15 GE 4 AND RAEL_Prof = 1 then RAEL_Pr_GE4 = 1; 

else RAEL_Pr_GE4 = 0; *Counting Hi-level proficienct RAELs; 

IF RAEL = 1 AND RAEL_Prof NE 1 AND CPL15 GE 4 then RAEL_NPr_GE4 = 1; 

else RAEL_NPr_GE4 = 0; *High-level non-proficient RAELs get the baseline; 

See alentaeiavediandeta EL SubGroup Code ERELEL EERE ES TERE A LEER EER ES RE 
data State_X_join; set State_X_join; 

if E016 = 1 then delete; *Activate this to calculate models for EL subgroup only; 

run; 


[EPREAREARASL EASA SELLA LAEAREAEALS EASELS EARLS EE EAEA LEASES REESE EES EARLE AS EERE AEE 


[= AGGREGATING TO THE SCHOOL LEVEL */ 


[ERAS ERARESLS ERLE EES LERER EERE EL ER EER EERE EEL ERES AL ESSE ES ELE EE EARLE LEELA ERLE S ERE 


/* Analysis below for ALL students EOs, ELs, and RAELs only NOT the EL Subgroup */ 
proc univariate data = State_X_join noprint; 
class school_number_2016; 
* these variables will be aggregated to the school-level and used in model calculations below; 
var ELA_Prof_16 
RAEL RAEL_Prof RAEL_lvl_change RAEL_pctl_change RAEL_perf_resid RAEL_ge4 RAEL_LT4 
RAEL_Gr_NPr_GE4 RAEL_Pr_LT4 RAEL_NPr_LT4 RAEL_Gr_NPr_LT4 RAEL_Pr_GE4 RAEL_NPr_GE4; 
* name of the School-level datatset; 
output out = State_X_school 
mean = mean_ela16 
sum = tot_prof16 tot_RAEL tot_RAEL_prof tot_RAEL_lvl_change tot_RAEL_pctl_change 
tot_RAEL_perf_resid tot_RAEL_GE4 tot_RAEL_LT4 
tot_RAEL_Gr_NPr_GE4 tot_RAEL_Pr_LT4 tot_RAEL_NPr_LT4 tot_RAEL_Gr_NPr_LT4 tot_RAEL_Pr_GE4 
tot_RAEL_NPr_GE4 
nobs = n_obs16; 
where grade16 NE .; 
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*the variable names starting with "tot_" reflect the total number of RAELs within a school in respective 
categories; 

* for example, tot_RAEL is the number of RAEL students within a given school; 

* tot_RAEL_Pr_LT4 is the number of RAELs that are proficient in ELA (in 2015-16) and below CPL 4.0 (in 2014- 
15); 

run; 


data State_X_models; set State_X_school; *Calculate outcomes for all Models; 


Mod0_ Baseline = tot_prof16/n_obs16; 
Mod2_Growth_Lvl_Change = (tot_prof16 - tot_RAEL_prof + tot_RAEL_Ivl_change)/ n_obs16; 
*(All proficient - RAEL Prof + All RAELs that changed level) / ALL; 


Mod2_Growth_Pct_Change = (tot_prof16 - tot_RAEL_prof + tot_RAEL_pctl_change)/ n_obs16; 
*(All proficient - RAEL prof + All RAELs that made 40th %) / ALL; 


Mod2_Growth_Resid = (tot_prof16 - tot_RAEL_prof + tot_RAEL_perf_resid)/ n_obs16; 
*(All proficient - RAEL Prof + All RAELs that made above-average growth) / ALL; 


Mod3_Op1 = (tot_prof16 + tot_RAEL_Gr_NPr_GE4)/(n_obs16); *(Total_proficient + High-level RAELs that 
made growth that were not proficient)/total; 


Mod3_Op2 = (tot_prof16 + tot_RAEL_Gr_NPr_LT4)/(n_obs16); *(Total_proficient + Low-level RAELs that 
made growth that were not proficient)/total; 


*It may also be desired to run analyses by RAEL numbers in schools; 


*Define School Size; 

data State_X_models; set State_X_models; 

if n_obs16 LT 10 then sc_size = 1; 

else if n_obs16 GE 10 AND n_obs16 LT 50 then sc_size = 2; 
else if n_obs16 GE 50 AND n_obs16 LT 100 then sc_size = 3; 
else if n_obs16 GE 100 then sc_size = 4; 

run; 


* Model comparison for all schools — here correlation and output; 

proc sort data = State_X_models; by sc_size; run; 

ods rtf file="[destination/name].rtf"; 

proc corr data = State_X_models; 

var ModO_Baseline Mod2_Growth_Lvl_Change Mod2_Growth_Pct_Change Mod2_Growth_Resid Mod3_Op1 
Mod3_Op2; 

run; title; 

ods rtf close; 


* Creating output that is accessible in Excel; 

ods csv file="[destination/name].csv"; 

proc corr data = State_X_models; 

var Mod0O_Baseline Mod2_Growth_Lvl_Change Mod2_Growth_Pct_Change Mod2_Growth_Resid Mod3_Op1 
Mod3_Op2; 
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by sc_size; 
run; 
ods csv close; 


[ERPERESES ER EASES SERA ELA CEES ELER EERE E SLES ES EEE RER REESE EEE ELLE ARE ERE RE LER SER ERER ESE ES / 


/* Analysis for SIFE Students */ 


[EEX SAESAERLER AL SEALER ER EE ALESE EERE LSE EES ERLE ELE REEL ER ALLEL EEL EEL ELE SEE SELES RE ER SAE 


ods csv file="[destination/name].rtf"; 
* SIFE Students Descriptives; 
* Counts; 
title "SIFE Count"; 
proc freq data = state_X_join; table RAEL * sife_status_2015; run; 
* SIFE by ELA variables; 
title "SIFE by ELA variables"; 
proc means data = state_X_join; class sife_status_2015; 
label ELA_SS_16=ELA SS; label ELA_Prof_16=ELA Prof; label Ivl_change=ELA ValTabl; 
label RAEL_pctl_change=ELA Pctl Growth; label perform=ELA Resid Gain; 
var ELA_SS_16 ELA_Prof_16lvl_change RAEL_pctl_change perform; 
run; 
title "SIFE by ELA variables and RAEL status"; 
* SIFE by ELA variables and RAEL status; 
proc means data = state_X_join; class sife_status_2015 rael ; 
label ELA_SS_16=ELA SS; label ELA_Prof_16=ELA Prof; label Ivl_change=ELA ValTabl; 
label RAEL_pctl_change=ELA Pct! Growth; label perform=ELA Resid Gain; 
var ELA_SS_16 ELA _Prof_16 lvl_change RAEL_pctl_change perform; 
run; 
* SIFE by ELA variables and ELP level; 
title "SIFE by ELA variables and ELP level"; 
proc means data = state_X_join; class CPL15 sife_status_ 2015; 
label ELA_SS_16=ELA SS; label ELA_Prof_16=ELA Prof; label lvl_change=ELA ValTabl; 
label RAEL_pctl_change=ELA Pct! Growth; label perform=ELA Resid Gain; 
var ELA_SS_16 ELA_Prof_16 lvl_change RAEL_pctl_change perform; 
run; 
ods rtf close; 
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